Template-free Articulated Neural Point Clouds for Reposable View Synthesis

Dynamic Neural Radiance Fields (NeRFs) achieve remarkable visual quality when synthesizing novel views of time-evolving 3D scenes. However, the common reliance on backward deformation fields makes reanimation of the captured object poses challenging. Moreover, the state of the art dynamic models are often limited by low visual fidelity, long reconstruction time or specificity to narrow application domains. In this paper, we present a novel method utilizing a point-based representation and Linear Blend Skinning (LBS) to jointly learn a Dynamic NeRF and an associated skeletal model from even sparse multi-view video. Our forward-warping approach achieves state-of-the-art visual fidelity when synthesizing novel views and poses while significantly reducing the necessary learning time when compared to existing work. We demonstrate the versatility of our representation on a variety of articulated objects from common datasets and obtain reposable 3D reconstructions without the need of object-specific skeletal templates. Code will be made available at https://github.com/lukasuz/Articulated-Point-NeRF

GazeStereo3D: seamless disparity manipulations

Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied

Effective user studies in computer graphics

User studies are a useful tool for researchers, allowing them to collect data on how users perceive, interact with and process different types of sensory information. If planned in advance, user experiments can be leveraged in every stage of a research project, from early design, prototyping and feature exploration to applied proofs of concept, passing through validation and data collection for model training. User studies can provide the researcher with different types of information depending on the chosen methodology: user performance metrics, surveys and interviews, field studies, physiological data, etc. Considering human perception and other cognitive processes is particularly important in computer graphics, where most research produces outputs whose ultimate purpose is to be seen or perceived by a human. Being able to measure in an objective and systematic way how the information we generate is integrated into the representational space humans create to situate themselves in the world means that researchers will have more information to implement optimal algorithms, tools and techniques. In this tutorial we will give an overview of good practices for user studies in computer graphics with a particular focus on virtual reality use cases. We will cover the basics on how to design, carry out and analyze good user studies, as well as different particularities to be taken into account in immersive environments.Peer ReviewedPostprint (published version

Non-rigid Transformations for Musculoskeletal Model

Vytvoření Virtual Physiological Human, fyziologického modelu lidského těla, vyžaduje registrace různých datových setů. Lze rovněž využít morfingu pro interpolaci nových dat. Tato práce se zaměřuje na transformace spojené s těmito operacemi a snaží se najít automatické řešení, které nevyžaduje zásah uživatele. Jako testovací aplikace bylo vybráno předzpracování modelů svalů pro deformační filtr v muskuloskeletálním modelu lidského těla vyvinutý v předchozí práci, který má problémy s poškozenými vstupními sítěmi, zejména s nemanifoldními hranami a vrcholy. Proto je cílem automatická oprava těchto sítí vzájemnou kombinací několika vstupních souborů získaných z různých zdrojů a to metodou multi-morphingu. Aby toto bylo možné, jsou zkoumány možnosti jejich vzájemné registrace, hledány vhodné parametrické domény a zvolen způsob jejich konečné interpolace. Následně je navrženo řešení, které zmíněné činnosti provádí pro obecné poškozené vstupní sítě se vzájemně podobným tvarem odpovídajícím zobrazovanému objektu, ale s obecnou výchozí polohou a s předem neznámým počtem topologických chyb, jako jsou zmíněné nemanifoldní hrany, vrcholy, ale i díry a izolované komponenty. Výsledek implementace návrhu je v závěru práce detailně testován a výsledky vyhodnoceny.Katedra informatiky a výpočetní technikyObhájenoThe Virtual Physiological Human initiative states importance of registration of data sets for creation of the model of the human body. It also mentions usage of morphing technique for interpolation of new data. This thesis focuses on the transformations tied with these operations and tries to find an automatic solution which does not need user set up parameters. The deformation filter for surface models of muscles in musculoskeletal model of human body developed in the previous work was chosen as testing application. It has difficulties with damaged input meshes, especially those containing non-manifold edges and vertices. Therefore, the goal is an automatic detection and removal of such artifacts, and the combination of several such inputs into one finer mesh surface gained using a multi-morphing method. To make this possible, approaches for mutual registration of input meshes are analysed, a suitable parametric domain is searched for and appropriate way of final interpolation is chosen. A solution for making such actions in fully automatic manner for general damaged input meshes with similar shape specified by underlying real-world object, but with various initial position and unknown number of topological artifacts consisting of holes and isolated components on top of previously mentioned non-manifold edges and vertices, is then suggested. The resulting method is then implemented and both partial steps and complete design is tested in various experiments. The results are discussed and conclusion is stated at the end of this thesis